Method For Centroiding And Tracking A Distorted Optical Image

ABSTRACT

A centroiding method is provided for an optical tracking system including a laser used for countermeasuring purposes in which a pencil thin laser beam is accurately positioned onto a target through the use of centroiding techniques for ascertaining the position not only of the target but also the laser beam, with the centroiding techniques resulting in a sub-pixel level resolution. The sub-pixel resolution permits utilization of smaller cost-effective focal plane arrays by giving the small focal plane array a resolution associated with much larger focal plane arrays.

RELATED APPLICATIONS

This application is a divisional of Ser. No. 12/228,511 filed Aug. 11,2008 which claims rights under 35 USC §119(e) from U.S. ProvisionalApplication Ser. No. 61/010,495 filed Jan. 7, 2008, the contents ofwhich are incorporated herein by reference.

STATEMENT OF GOVERNMENT INTEREST

This invention was made with United States Government support underContract No. HSSCHQ-04-C-00342 awarded by the Department of HomelandSecurity. The United States Government has certain rights in thisinvention.

FIELD OF THE INVENTION

This invention relates to providing countermeasure energy in a verytight pencil beam onto an incoming missile target, and more particularlyto a method for precisely pointing the very tight pencil beam onto thetarget.

BACKGROUND

COUNTERMANPAD Applications abound in which countermeasures are used todeflect or defeat an incoming shoulder-fired missile. In order to divertor defeat the incoming missile, infrared laser radiation is projectedtowards the target with a so-called jam code which confuses the seekerin the missile and causes the missile to be diverted away from itsintended target.

Countermeasuring incoming missiles can be accomplished utilizing flares,and if the incoming missile is a heat-seeking missile, then theheat-seeking missile will follow all the flares as opposed to theoriginal target.

However, if the target is large, meaning it has large bright engineswhich produce a significant IR signature, then the incoming missile willhome in on the rather large thermal signature from the aircraft, such asa 747. If an IR countermeasure laser is used to countermeasure themissile and if its beam is too wide, there will not be enough energy onthe missile's seeker and the missile will still home in on the brightengine. Thus, there is a requirement for a pencil thin laser beam toconcentrate sufficiently high energy onto the missile's seeker.

If one had an infinitely high-power laser, one could tolerate a widebeam. However, this is not practical and in order to minimize laserpower as well as size, it is more appropriate to generate a pencil-thinlaser beam which concentrates its energy in a very small but intensespot. The intensity of this laser beam is such that it completely drownsout the thermal signature of a large target.

However, when generating a pencil-thin laser beam, pointing accuracy isof utmost importance. For instance, for such a pencil-thin laser beamthere is a requirement that the pointing accuracy be less than adetector's Instantaneous-Field-of-View (IFOV), which at 8 miles is ofless than 4 feet.

Ascertaining where the target is to this pointing accuracy is extremelydifficult because the IR image of the target is blurred out on the focalplane array normally used and overlaps numbers of pixels. Normally, theresolution is limited to the IFOV of a pixel. For a Focal Plane Array(FPA), the angular error is associated with the size of a pixel, whereasthe requirement is on the order of ¼ of a pixel's IFOV.

While auto boresighting techniques are utilized to establish the exactposition of the laser beam in space, the center of energy of the blur isdifficult to ascertain, especially when utilizing a focal point of arrayhaving a resolution limited by the pixel IFOV.

It is therefore important to be able to ascertain laser pointingdirection and the IR image center to an accuracy better than thatassociated with the IFOV of a pixel.

As will be appreciated with a very tight pencil-thin laser beam used forconcentrating energy in a very small cross-section, it is exceedinglyimportant to be able to take the blurred out IR missile plume imagewhich can overlap a number of pixels on the focal plane array anddetermine the exact point at which the laser beam is to be aimed toaccuracies greater than the pixel level resolution of the array.

More particularly, detector arrays used in countermeasure systems employa focal plane array onto which energy is imaged. The resolution of thefocal plane array is determined by the size of the array and the pixelsize. For very large arrays, such as 1028×1028 arrays, pixel size isvery small and resolution is only limited by this small pixel size. Insome cases pixel size can be under 10 microns and with improved opticsthe images can be focused down to such a small spot.

However, these large arrays are costly and exceed weight constraints.More importantly, the read out of so many pixels simultaneously or inrapid succession is difficult and requires a considerable amount ofprocessing. There is therefore a need to provide the same resolution asa large array in a small affordable 128×128 array. To do this one needssub-pixel resolution.

All things being equal, the size of the pixels determine the effectivepixel IFOV and up until the present time, the resolution that determinesthe location of the target has been limited by the inability of smallarrays to resolve focal plane array images below pixel level.

To put it another way, depending on how much jam energy one wants to puton the target, the efficiency of the jamming is related to how large theaircraft is, because the larger the aircraft, the brighter the engine.The brighter the engine, the more energy that must be focused onto thetarget to sufficiently increase the jam-to-signal ratio or J to S.Depending on the platform one uses, one has to generate a considerable Jto S for bright engines.

One can either spread energy out into the environment like a bigflashlight beam, but it is very weak compared to the bright engine.Thus, in order to project sufficient energy onto the target, one has toshrink the beam to pencil thinness, thus to concentrate the energy intoa small spot. This means that track accuracy is very tight or the spotwill miss the target.

SUMMARY OF INVENTION

In order to reduce the tracking error and to overcome the resolutionassociated with a small focal plane array, the subject inventioncalculates the center of energy or centroid of the IR image, namely therocket plume, and utilizes this center-of-energy point as the aimingpoint for the system.

It has been found with the subject algorithm, that one can locate thecenter of energy of the target image by interpolation to an accuracybetter than 0.25 of a pixel. This means that one can narrow theeffective IFOV of a pixel and thus determine target position moreaccurately; and in fact accurately enough to put the pencil thin laserbeam directly on the missile seeker head.

The ability to interpolate to better than a pixel level is dependentupon the signal-to-noise ratio of the system, with the larger thesignal-to-noise ratio the better the subject interpolation algorithmworks.

In one embodiment, the subject interpolation algorithm is utilized bothto ascertain the centroid of the laser beam, which is quite narrow tobegin with, and then applies the same algorithm to detect the centroidof the infrared target image.

Using the subject technique, the point at which the laser is aimed inspace can be ascertained to accuracies in excess of pixel levelaccuracy, as can be the center of energy of the infrared target image.

Having successfully ascertained the laser pointing direction utilizingthe subject technique, and having then used the subject algorithm toascertain the center of energy of the infrared target image, one candrive the center of energy of the target image to coincide with thepointing direction of the laser. At this point, the laser beam will beprojected onto the target with an error corresponding to a 0.25 pixeleffective IFOV.

The algorithm describing the detection of the centroid of the laser beamor IR target image will be described hereinafter.

Essential for the subject algorithm is the overlapping of the image witha number of pixels, and this is normally the case. If the image is tootightly focused it will not overlap adjoining pixels. If such is thecase, the system is set up with the focal plane array slightly offsetfrom the focal plane to provide the necessary defocus.

To provide sub-pixel level resolution in a small array, in essence, thesubject algorithm considers only a portion of the focal point array, forinstance a 3×3 array. The algorithm then obtains the values of theoverlapping image along three vertical lines corresponding in positionto the center of the respective pixels. Those values are summed anddivided by the sum of the values to provide an average value in the Xdirection. Likewise, in the Y direction, for horizontal rows runningthrough the centers of the three pixels, one can determine by thesubject averaging technique, the average position of the image in the Ydirection.

This being the case, it is possible to obtain the position of thecentroid of the image to an accuracy less than a 0.25 pixel width, thusto give a laser pointing error of less than 0.25 pixel effective IFOV.This permits accurate determination of both the position of the laserbeam and the position of the target to such accuracy that a thin laserbeam can be projected directly onto the target.

More particularly, the subject algorithm includes the following steps.

-   -   1. Seed the algorithm with the brightest pixel in the image;    -   2. Make sure that the brightest pixel is far away from the edge        to perform the centroiding process;    -   3. Determine the average background around the brightest pixel        to be centroided;    -   4. Limit the area to be centroided;    -   5. Calculate the target intensity by summing intensities on the        centroided area;    -   6. Remove the background from the target intensity;    -   7. Calculate the offset for the centroid in the X direction,        which is each row's intensities in the centroid is multiplied by        a multiplier;    -   8. Calculate the row coordinate for the centroid by adding the        weighted sum to the current row and dividing by the total target        intensity;    -   9. Calculate the offset for the centroid in the Y direction in        which each column's intensities in the centroided area is        multiplied by a multiplier; and    -   10. Calculate the column coordinate for the centroid by adding        the weighted sum and dividing by the total target intensity.

What is now presented is the centroid equation set that implements theabove algorithm:

Centroid Equation Set Output Equations are $\quad\begin{matrix}{{row\_ centroid} = {{row\_ seed} + \left( \frac{{weighted\_ sum}{``{row}"}}{total\_ intensity} \right)}} \\{{col\_ centroid} = {{col\_ seed} + \left( \frac{{weighted\_ sum}{``{col}"}}{total\_ intensity} \right)}}\end{matrix}$ The seed is the pixel having the highest energy. The“weighted sum” terms are generated by the following summations:${weighted\_ sum} = {{\sum\limits_{I_{CSR}}^{I_{CER}}\left( {\sum\limits_{I_{CSC}}^{I_{CEC}}{{P\left( {{row},{col}} \right)}\mspace{14mu} {total\_ background}}} \right)} - {multiplier}}$Reversing the row and columns summations changes the weighted sum toapply to either row or column. The multiplier term is computed by anoffset against an arbitrary value called “half centroid size” GivenI_(CER) ≡ “Centroid End Row” Given I_(CSR) ≡ “Centroid Start Row” GivenI_(CEC) ≡ “Centroid End Column” Given I_(CSC) ≡ “Centroid Start Column”Given P ≡ Pixel Intensity as function row and column Then${Total\_ Intensity} = {\sum\limits_{I_{CSR}}^{I_{CER}}{\sum\limits_{I_{CSC}}^{I_{CEC}}{P\left( {{row},{column}} \right)}}}$Lastly the “total_Intensity” is itself just a summation about the seedpixel

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features of the subject invention will be betterunderstood in connection with the Detailed Description in conjunctionwith the Drawings, of which:

FIG. 1 is a diagrammatic illustration of the use of a pencil thinjamming beam against an incoming threat, indicating that the requiredaiming accuracy can be achieved by applying the subject technique tobring down the effective instantaneous field of view of a pixel to 0.25IFOV;

FIG. 2 is a block diagram illustrating the subject countermeasuringsystem, including a modulated laser, a focal plane array, and an IRtarget image, along with the utilization of a laser beam combiner;

FIG. 3 is a diagrammatic illustration of an auto boresighted laser beamposition on a 3×3 focal plane array, also illustrating the detectedcenter of energy of an image blur overlapping multiple pixels, with thecenter of energy of the blur being moved to correspond to the laserboresight direction, also illustrating the resolution of the focal planearray to be that associated with one pixel;

FIG. 4 is a diagrammatic illustration of the subject interpolationalgorithm for determining the center of the image blur, indicatingsumming of energy along vertical columns and horizontal rows which passthrough the centers of associated pixels;

FIG. 5 is a diagrammatic illustration of the determination of the laserbeam direction utilizing the subject interpolation algorithm to find thecenter of energy of a blurred target image;

FIG. 6 is a diagrammatic illustration of one embodiment of the subjectinvention, including a focal plane array, a laser and an opticalparametric oscillator in which the output of the optical parametricoscillator is directed to a laser beam combiner both for projection outinto the environment of the laser beam and for laser boresightingpurposes; and,

FIGS. 7A and B are graphs of tracker centroid errors versussignal-to-noise ratio for a blurred IR image and the image of the laserof FIG. 6, also showing three-dimensional image intensity graphscorresponding, respectively, to the blurred infrared image and the laserimage on the focal plane array.

DETAILED DESCRIPTION

Referring now to FIG. 1, an incoming missile 10 having a seeker head 12is shown traveling towards its intended target. In order tocountermeasure the missile, a pencil thin laser beam 14 is directedtowards the missile head. This beam carries a jam code that includes ajamming sequence. The position in space of the pencil thin beam needs tobe directly on seeker head 12 to an accuracy of less than a pixel IFOV.Should the angular error exceed more than a pixel IFOV, it is possiblethat the pencil thin beam may miss its target completely.

More particularly, the effective instantaneous field of view (IFOV) of asinge pixel 16 is shown at 17 and subtends an angle greater than onewhich would intercept the missile. However, by techniques describedherein, despite pixel overlap of the IR target image, the effectiveresolution of a single pixel can be increased fourfold. This means thatthe effective IFOV 17 can be cut by ¼ to subtend a 0.25 effective IFOVangle as shown at 18. The result with this accuracy is that a pencilthin laser beam spot will hit jam head 12.

Referring now to FIG. 2, a typical countermeasure system includes alaser 20 modulated with a jam code 22, projected towards a laser beamcombiner 24, which redirects the beam from laser 20 to out along axis 26towards an incoming missile 28.

The position of optical axis 26, and therefore the direction of theoutgoing beam, is determined by directing a portion of the laser outputtowards a focal plane array 30, from which the position in space of theoutgoing beam is ascertained using auto boresighting techniques.

Thereafter, an IR image 32 of the hot rocket exhaust of missile 28 isimaged onto the focal plane array so that the position of the incomingmissile may be ascertained.

The laser pointing head is gimballed such that the laser beam from laser20 is centered on the infrared target image. This is accomplished byimaging the laser beam on the focal plane array and moving the infraredtarget image detected by the focal plane array to the sensed position ofthe laser beam.

However, as mentioned above, for small arrays the infrared image of theplume from the target extends over multiple pixels on the focal planearray. Its center of energy, which determines the point in space wherethe missile is located must be ascertained, typically to a resolutionbetter than the single pixel resolution of the array. In one embodiment,the subject system improves this single pixel resolution fourfold.

Referring to FIG. 3, a portion of the focal plane array, here a 3×3portion, is indicated at 40. Auto boresighting techniques andcorrections to be discussed herein provide the precise and correctedposition 42 of the laser beam. Also indicated is the detected IR image44, which as can be seen straddles four pixels, namely pixels 46, 48, 50and 52 of array 40.

It is the purpose of the interpolation algorithm to find the center ofenergy of blur 44, here illustrated at 54. As illustrated by arrow 56,this permits moving the detected center of energy to coincide with theauto boresight position. This is done by gimbaling the laser to move thedirection of the projected laser beam to end up on the target whoseposition is ascertained by the detected center of energy of the blur.Put another way, the detected center of energy of the blur is moved tobe coincident with the auto boresight corrected position 42.

As can be seen at 58, in one embodiment the target blur is smeared outin a Gaussean fashion, such that it overlaps more than one pixel. Theresolution of the array is determined by the pixel size of pixel 60, inone embodiment 30 microns on a side.

It is the purpose of the subject invention to provide aiming accuracybetter than a single pixel IFOV, at least for a 128×128 array having 30micron pixels.

Referring now to FIG. 4, how the center of energy of the blur for thetarget is ascertained as follows:

For the X direction, one measures detected radiation along threecolumns, x₁, x₂ x₃, which are centered on the respective pixels throughwhich the columns pass.

Likewise, in the Y direction, rows y₁, y₂, and y₃ pass through thecenters of the corresponding pixels.

The radiation incident on vertically-spaced pixels along columns x₁, x₂,and x₃ is summed, with the sum of the sums being divided by the sum ofall of the detected radiation on the pixels of the array. The same istrue for the Y direction.

This scenario has been described by the aforementioned centroidingequations. These equations locate the center of energy of the detectedblurred image so that even though the blur overlaps numbers of pixels,its center can be ascertained to less than a pixel width in accuracy.

Referring to FIG. 5, the same technique is utilized to determine thecenter of energy of the outgoing laser beam, which is shown at 66 tooverlap at least three pixels.

While the image of the laser beam on the focal plane array is quite abit smaller than the blurred image of the target, it is important in thesubject invention to be able to ascertain with sub-pixel level accuracyits position.

Thus, when the detected center of energy of the target blur is madecoincident with the center of energy of the target laser beam, less thanpixel level accuracy is achieved.

Referring now to FIG. 6 in one operative embodiment, a focal plane array70 with a 1×1 detection kernel with guardband is positioned behindoptical assembly 72 which focuses infrared radiation 74 from a missile76 onto the focal plane array. Here, sensor processor assembly 78processes the output of the focal plane array.

Laser 80 is coupled to an optical parametric oscillator (OPO) 82 via afiber optic interface 84, with the output of the OPO being re-directedby a prism 86 in a module 88 through a laser beam combiner 90 and onto are-directing prism 92, which re-directs the laser beam back to the laserbeam combiner and then to the focal plane array. The position of thelaser beam image on the focal plane array specifies the aiming directionof the laser, and this position is refined by the subject interpolationalgorithm.

The position of laser beam 94 in space is thus first determined bypicking off a portion of the laser beam from the optical parametricoscillator and determining its position, again through the utilizationof the subject interpolation algorithm.

Likewise, the infrared return from missile 76 passes through the laserbeam combiner 90 and onto the focal plane array as the aforementionedblur.

As will be discussed hereinafter, the utilization of the laser beamcombiner degrades the signal-to-noise ratio of the system due toaperture blockage. To the extent that this blockage is minimized thesignal-to-noise ratio is increased. The above reduction of the effectivesingle pixel IFOV in one embodiment requires a signal-to-noise ratiogreater than 20.

Referring now to FIGS. 7A and 7B, what is seen are graphs of the IRtracking error and laser boresight error graphed against signal-to-noiseratio.

To the right of FIG. 7A is a three-dimensional representation of thetarget blur intensity indicating a fairly pronounced center region andside lobes.

It can be seen from the graph of FIG. 7A that a signal-to-noise ratio of20 provides a tracker error of less than one pixel width, and in thiscase 0.25 of the pixel, leading to an effective 0.25 IFOV for the pixel.

Note that if the signal-to-noise ratio less than about 8, there is noadvantage with the subject interpolating algorithm.

Likewise, referring to FIG. 7B, although the laser beam has aconsiderably better defined cross-section and shape, with asignal-to-noise ratio of about 100, the tracker/laser boresight error isagain less than one pixel width, and in this case is also 0.25 of thepixel.

The algorithm for interpolation is provided below.

function [row_centroid,col_centroid] =determine_target_centroid(data,row_seed,col_seed) % Target Centroidcalculation; % initial values for track error %  background_width = 7; % half_centroid_size = 2; %  centroid_width =3; % initial values forlaser (boresight error) background_width = 29; half_centroid_size = 2;centroid_width =25; % maxval = max(max(data)); % maxval =max(max(data(29:35,29:35))); % given the seed which is the brightestpixel in the image.We will use that one for the % centroidbrightest_pixel_row = round(row_seed); brightest_pixel_col =round(col_seed); brightest_pixel_intensity =data(brightest_pixel_row,brightest_pixel_col); % Make sure the brightestpixel is far away from the edge to perform the % centroid. This is thefunction of the size of the centroid [obstructed_edge,row,col] =Check_detection_edges(brightest_pixel_row,...brightest_pixel_col,background_width,half_centroid_size,data); % Nowdetermine the average background around the hit to be centroided[average_background] =determine_target_background(brightest_pixel_row,...brightest_pixel_col,obstructed_edge,half_centroid_size,data,centroid_width); total_background = average_background * centroid_width;% limits of the area to centroid centroid_start_row =brightest_pixel_row − half_centroid_size; centroid_end_row =brightest_pixel_row + half_centroid_size; centroid_start_col =brightest_pixel_col − half_centroid_size; centroid_end_col =brightest_pixel_col + half_centroid_size; % Total target intensity iscalculated by summimg the intensities on the % centroid areatotal_intensity =0; for i = centroid_start_row : centroid_end_row   forj = centroid_start_col : centroid_end_col     pixel_intensity =data(i,j);     total_intensity = total_intensity + pixel_intensity;    pixel_array(i,j) = pixel_intensity;   end end % Removing thebackground from the target intensity total_intensity = total_intensity −average_background; if total_intensity > 0   weighted_sum = 0;  multiplier = − half_centroid_size;   % calculation of the offset forthe centroid   % each rows intensities in centroid area are multipliedby a multiplier   for i = centroid_start_row : centroid_end_row    weighted_row =0;     for j=centroid_start_col : centroid_end_col      weighted_row = weighted_row + pixel_array(i,j);     end    weighted_sum = weighted_sum + (weighted_row−total_background)*multiplier;     multiplier = multiplier +1;   end   %the row coordinate for centroid is calculated by adding weighted sum   %to current row and dividing up by the total target intensity  row_centroid = row_seed + (weighted_sum/total_intensity);  weighted_sum = 0;   multiplier = − half_centroid_size;   % eachcolumns intensities in centroid area are multiplied by a multiplier  for i = centroid_start_col : centroid_end_col     weighted_col =0;    for j=centroid_start_row : centroid_end_row       weighted_col =weighted_col + pixel_array(j,i);     end     weighted_sum =weighted_sum + (weighted_col− total_background)*multiplier;    multiplier = multiplier +1;   end   % the column coordinate forcentroid is calculated by adding weighted sum   % to current column anddividing up by the total target intensity   col_centroid = col_seed + (weighted_sum/total_intensity); else   row_centroid =brightest_pixel_row;   col_centroid = brightest_pixel_col; end

It can therefore be seen that the laser aiming accuracy is criticallydependent upon reducing the effective pixel IFOV which improves theresolution of the focal plane array, thus to more accurately direct thepencil-thin laser beam onto the target.

1. A method for improving the resolution of a focal plane array used inan optical tracking system to sub-pixel resolution, comprising the stepsof: processing the output of the focal plane array using centroiding toascertain the center of images on the focal plane array that overlapmultiple pixels for establishing the center of energy of the image onthe focal plane array to sub-pixel level resolution; and, utilizing thecentroided image information to establish the position of an infraredimage on the focal plane array.
 2. The method of claim 1, wherein theinfrared image includes a target image and wherein the optical trackingsystem includes a laser modulated with a jam code for countermeasuringthe target.
 3. The method of claim 2, wherein the optical trackingsystem includes using centroiding to detect by a portion of the image ofthe laser beam on the focal plane array the pointing direction of thelaser beam.
 4. The method of claim 3, and further including the step ofdetermining to a sub-pixel level the position of the infrared targetimage and to a sub-pixel level the position of the portion of a laserbeam, and, changing the position of the laser beam so as to effectuatecoincidence between the centroided position of the target image and thecentroided position of the laser beam, whereby sub-pixel resolution isafforded to the optical tracking system.
 5. The method of claim 4,wherein the sub-pixel level resolution results in a pointing accuracy ofless than the number of microradians associated with sub-pixel levelaccuracy.
 6. The method of claim 5, wherein the sub-pixel levelresolution is less than a 0.25 IFOV.
 7. The method of claim 1, whereinthe focal plane array is a 128×128 array or a 256×256 array.
 8. Themethod of claim 7, wherein the 128×128 or 256×256 array includes 30micron pixels.